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The comparative effects of instruction in 
self-questioning, summarizing, and question-answer relations on 
reading comprehension test performance were studied with 49 juniors 
nd seniors enrolled in a remedial reading course at Georgia State 
University. The objective was to find a method to help students pass 
a statewide college reading exit examination. In the item-writing 
training procedure, students were taught to write multiple-choice 
comprehension questions based on their reading, while the summary 
writing training followed Day's summarization rules. Finally, 
students were introduced to the taxonomy of question-answer relations 
using the "Mill Mends His May" passage and questions from Pearson and 
Johnson (1978). All students received 4 to 5 hours of instruction and 
practice per week for a month within the context of the class. Three 
evaluation measures were employed: the Basic Mord Vocabulary Test 
during the first week, a practice form of the state exit reading 
comprehension examination during the last week, and the actual 
Regent's Reading examination. Students trained in discrening 
question-answer relations were more successful on reading 
comprehension tests than students trained in either summarizing or 
item writing. (SM) 
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i Training Effects of Summarizing, Item Writing and Knowledge 

i of Information Sources on Reading Test Performance 

> 

Metacognitive processes during reading such as summarizing 

« 

■ and sel f -quest ioning have been shown to be effective in 

transferring information into long-term recall (Flavell & 

t 

i Wellman, 1977) particularly when the reader processes the "right 

\ information" in the "right way" (Anderson & Armbruster, 1980). 

; The application of comprehension monitoring techniques to the 

j process of answering reading comprehension test questions has 

j been advocated by Pearson and Johnson (1978) and Raphael (1984) 

« in the form of awareness of question-answer relations. The 

question we have posed for this study is: What are the 
comparative effects of instruction in summarizing, self- 
4 questioning and qu e s t i on - a ns we r relations on reading 

comprehension test performance? 
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Review of Research 

An essential element of effective studying is the ability to 
estimate one's readiness to be tested. Two commonly reported 
sophisticated methods of testing one's level of comprehension and 
retention and, therefore, one's preparedness for a test, are to 
attempt to summarize or to question one's self over the material 
one has been reading (Brown, Campione & Day, 1981). A third, 
less common approach to test preparation is to develop one's 
sensitivity to sources of information for answering comprehension 
questions (Raphael, 1984). According to Armbruster and Brown 
(1984), good readers monitor and evaluate their comprehension and 
retention relative to the demands of the criterion task. Poor 
readers, on the other hand, are much less aware of the self- 
regulatory strategies for learning efficiently and effectively 
from text. Brown (1981) has concluded from her work that 
metacognitive processes are developmental in nature and that 
secondary and college students can be taught systematically 
metacognitive skills to help them learn from text. 

With the development of the theories of metacognition and 
schema came major criticisms of previous research on 
questioning impelled by Rothkopf's mathemagenic hypothesis 
(1965). Critics contend that studying questioning as a text- 
inspection behavior is not fruitful- because it will reveal very 
little about the mental processes that underlie this behavior and 
that Rothkopf -1 ike questioning research lacks a theoretical 
foundation which would explicate the relationship between 
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cognitive processes and mathemagenics (Anderson & Biddle, 1975; 
Andre, 1979; Trabasso, 1981). Training in the skill of self- 
questioning during reading as a metacognitive process has shown 
promise (Andre & Anderson, 1978-79; Cohen, 1983; Wong & Jones, 
1982). Singer and Donlan (1982) were successful in teaching 
eleventh grade students to derive story-specific questions from 
schema-general questions as a means of developing problem solving 
schema for comprehending short stories. Combining training in 
self -questioning and summarizing, Bean, Singer, Sorter and Frazee 
(1983), however, were not as successful in teaching eleventh 
graders to generate their own questions from expository prose 
passages based on a knowledge base presumed to have been acquired 
through following Day's summarization rules. In a study of self 
questioning reported by Duell (1978), college students were 
provided training in writing multiple-choice questions based on 
content objectives at different cognitive levels. By providing 
students with objectives, they were made awar f the reading 
task demands and could monitor their question-production relative 
to the objectives. On the criterion test, subjects who received 
higher and lower level objectives performed better than a control 
group . 

Recent research with summarization has grown out of the 
theories of Kintsch and van Dijk (1978) and van Dijk and Kintsch 
(1978). Based on their postulation of three major rules for text 
comprehension and talk-aloud protocals of expert readers, Day 
£1980) developed summarization rules and found average and 
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remedial college students could be trained to use the rules. 
Specifically, she found that advanced students needed less 
explicit instruction than less advanced students, and that after 
training all subjects exhibited a greater use of the rules. 
Following this same line of research, a series of experiments 
conducted by Brown and Day (1983), Brown, Day, and Jones (1983), 
and Winograd (1984) provide further support for the developmental 
patterns that students of various ages demonstrate when preparing 
summaries. None of this research, however, was designed to 
determine whether training in making summaries would actually 
improve reading comprehension. More recently, King, Biggs and 
Lipsky (1984) compared the effectiveness of developing summaries 
or generating prequestions where the criterion tasks involved a 
free recall measure, an objective test and an essay test. While 
there were no significant differences between the treatment 
groups on the free recall measure and the objective test, the 
subjects undergoing the summarization training significantly 
outperformed their peers in the prequestioning group on the essay 
test of reading comprehension. 

In a pilot study for this research (Stahl, Brozo & Gordon, 
1983) remedial college students were trained to either create 
summaries according to Day's rules or construct multiple-choice 
quesc ns from a table of specifications. Although the findings 
were not significant, both treatment groups outperformed a 
control group, using a traditional workbook approach, on a 
college exit examination of reading comprehension. in the 
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present study, we compared the effects of training in summarizing 
and self-questioning with a third condition, training in locating 
information sources for answering reading questions (Pearson & 
Johnson, 1978). According to Gordon (1982), students can be 
taught to reason through comprehension questions by employing 
problem solving schema, In a series of studies by Raphael 
(1984), students were taught about sources of information for 
answering comprehension questions which both sensitized them to 
the task demands of questions and also improved their ability to 
answer questions. 

The purpose of the present study was to find an effective 
method which would improve students' chances for passing a 
statewide college reading exit examination. Given the short 
duration of our remedial course (five weeks) and the severe 
deficits of the participants, we are forced to either teach to 
the test, or rely on test-taking manuals or chapters from study- 
skills workbooks to teach test-taking skills. In either case, 
needed time for instruction in sound strategies foi reading and 
learning from text is compromised seriously. We trained students 
to make summaries, write multiple-choice questions and discern 
question answer relations and postulated they would transfer 
these metacognitive skills to the process of taking the exit 
reading examination and consequently improve their performance. 
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METHOD 

Materials and Procedures 

Item-writing training procedure (IW). Students under this 
treatment condition were trained to\ write multiple-choice 
comprehension questions based on their reading. "'he process, a 
variation of the work by Duell (1978), involved the following 
> steps: 

1. Identify important information from the passage. 
Students were taught to focus on main ideas and relevant 
supporting details and isolate these portions of text through the 
use of underlining and marginal cues. 

2. Build a simple table of specification which connects 
content with an appropriate question. Students were instructed 
to use a taxonomy of cognitive processing levels, namely literal, 
inferential and analytical, to match with the information 
identified in step one. 

3. Consult available examples of question stems as a guide 
for writing new stems. Students analyzed copious examples of 
other items and were directed to identify those cases in which 
the wording in many item stems was repeated from passage to 
passage . 

4. Develop a system for writing reponse options. Students 
model ,ed a system for generating distractors which drew upon 
incorrect information, irrelevant details and inconclusive ideas. 

5. Write questions and gather feedback. Questions were 
exchanged and evaluated by other students and the instructor on 
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the bases of appropriateness and accuracy. Examples of well- 
written and poorly-written questions were put on transparancies 
and discussed. 

The practice materials for the IW group were one page 
articles taken from a popular weekly news magazine. 

Summary writing training procedure (SW). Students in this 
group were taught to follow Day's summarization rules. The 
process involved following five general steps to help in summary 
writing and five specific rules to follow in writing tne actual 
summary. The general steps were: 

1. Make sure you understand the text. The student tries to 
say the general theme of the text to him/herself. 

2. Look back. The student rereads to reinforce 
understanding and to star important parts. 

(at this point the student begins to follow the rules for 
writing a summary) 

3. Rethink. The student rereads a paragraph of text to 
determine if the topic sentence was properly identified or 
written . 

4. Check. The student rechecks lists and categories to 
include elements that were left out excise redundant elements 
from the summary. 

Double check. The student conducts a final review of the 
summary to ensure inclusion of all important information. 
The specific rules for writing a summary were: 
1. Reduce lists. The student provides categories for 
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examples . 

2. Use a topic sentence if one is given you. The student 
identifies topic sentences in paragraphs and includes them in the 
summary . 

2. Make up your own topic sentence. The student writes a 
topic sentence for paragraphs without one and includes the topic 
sentences in the summary. 

4. Get rid of repeated information. The student crosses 
out repeated information in the text. 

5. Get rid of unimportant information. The student crosses 
out unimportant information in the text. 

Students in the SW group applied this strategy to the same 
one page articles used by the IW group. 

Question answer relations training procedure ( QAR ) . 
Students were introduced to the taxonomy of question-answer 
relations using the "Will Wends His Way" passage and questions 
from Pearson and Johnson (1978). Due to the non-explicit nature 
of many college level reading test questions and answers, the 
Text Explicit category of the taxonomy was expanded to include 
any item answerable from a single sentence in the text: those 
explicitly cued, those with synonym substitutions or paraphrases 
[Johnston's (1981) modification], and those with neither question 
stem nor answer explicitly cuJd. The categories of Textually 
Implicit and Scriptally Implicit were not modified. With the 
Pearson and Johnson passage, students were shown how different 
answers can be supplied for the same question depending on 
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whether the information sour'": came from the text or the readers 
background. 

After the introduction, students practiced the question 
system using passages from textbooks, newspapers and magazines 
with teacher-generated multiple-choice items. At this stage in 
the training they were provided with the correct answers prior to 
reading the passages since the purpose was to practice 
identifying information sources for answering the items, with 
the first passage, students were directed to (1) mark the 
information in the text they would use to obtain the correct* 
answer, (2) write down any prior knowledge neccesary to answer 
the item correctly and (3) classify each item. Responses were 
discussed at length, then the students were directed to folio* 
the same procedure for the remaining passages. These passages 
were collected, examined by the instructor to ensure that the 
studentsunderstoodthetechnique and then feedback wasprovided. 
The practice phase varied from the training phase only in that 
students, with new passages and items, had to answer the questions 
themselves. For any question answered incorrectly, the student 
was given the correct answer and directed to mark the information 
in the text and to provide the scriptal information when 
necessary. In the final week, students were not provided with 
the answers but were required to repeat the text marking/item 
classification procedure. 

Students in all three conditions were provided four to five 
hours of instruction and practice per week over a four week 

i 
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period witnin the context of the class. Daily assessment oi 
students' papers provided evidence that they were employing and 
had learned their respective strategies. 

Three measures were employed: (1) students completed the 
Basic Word Vocabulary Test [ (BWVT) (1984)] during the first week; 

(2) during the fifth and final week of the course, students were 
administered a Practice Form of the state-wide exit examination 
of reading comprehension, a timed, standardized test made up of 
ten passages followed by a total of 58 multiple choice questions; 

(3) at the conclusion of the course, students took the actual 
state-wide Regent's Reading examination, a parallel version of 
the Practice Form. Ctudents take this test outside the class in 
a formal setting under the supervision of a proctor, and test 
results are made avi?lable dnly in the form of scaled scores. 
Raw scores canno*: be obtained. 

Subjects 

Forty-nine junior and senior-level college students enrolled 
in a remedial reading course at Georgia State University 
participated in the study. They comprised two groups of 16 
students and one group of 17 students. The BWVT served as a 
covariax.e to statistically equalize the groups. 

RESULTS 

The first ANCOVA, with the BWVT as the covariate, was 
performed r first measure of transfer, the Practice Form. 

The QAR group achieved a significantly higher mean score than the 
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other two groups, F( 2 , 4 5 ) = 6. 1 3 , p<.01. Means and standard 
deviations for the Practice Form are displayed in Table 1. An 
additional ANCOVA was performed on the second measure of 
transfer, the Regent's Reading test. The QAR and IW groups 
outperformed the SW group, F(2,45)=5.54, p<.05; however, there 
was no significant difference between the QAR and IW groups (see 
..Mo 2) 
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DISCUSSION 

The study was designed to determine an efficient and 
effective method of preparing college students in a remedial 
reading course for a an exit examination of reading. Previous 
instructional practices had relied heavily on "how to" study 
skills manuals for test preparation or focused on teaching to the 
test while failing to inculcate reading strategies that promote 
independent problem-solving and sel f -evaluation. in a pilot 
study, training in summarizing and self -questioning in the form 
of item writing proved superior to the traditional approach. In 
this study support was found for considering training in question 
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answer relations over summarizing and item writing as a means of 
instruction in our course. On the first measure of transfer, 

f essentially a practice test for the actual Regent's reading test, 

the QAR group achieved significantly higher scores than both the 
IW and SW groups. On the actual exit examination, the Regent's, 

< the QAR group significantly outperformed the SW group only. 

We have a possible explanation for the findings relative to the 
QAR group which is based on the principal of encoding specificity 
(Tulving & Thomas, 1973). According to this principal, the way 

1 in which information is encoded determines how it is stored and 

* determines which retrieval cues will effectively access it. The 

v 
t 

°* implication of this principle for studying is that if the student 

knows the requirements of the criterion task and encodes the 
information in an optimal form to meet those requirements, task 
performance will be facilitated (Anderson & Armbruster, 1980). 
During training, the QAR group processed information in a form as 
close as possible to the requirements of the and transfer tasks. 
In fact, the sensitivity developed in the QAR group to 
information sources for answering comprehension questions was 
applied directly to the criterion and transfer tasks. This was 
not the case for the IW and SW groups. Neither could apply 
directly their newly learned processing skills due to the timed 
nature of both tests and especially under the controlled 
conditions in which the Regent's test is taken. The notion of 
encoding specifity helps explain the inferior performance of the 
SW group on the transfer tasks. King, et.al. 
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(1984) concluded that the effects of summary writing training can 
best be detected on tasks which resemble those required in 
training (written protocals, essays, summaries). Having students 
create summaries and write multiple-choice items may be requiring 
them to process the information in the "right way," that is, at a 
deep and meaningful level of involvement with the text; however, 
for this study, these text processing strategies could not be 
applied to the "right information," that is, on the transfer 
tasks. The evidence indicates that training in discerning 
question answer relations promotes comprehension skills necessary 
for good performance on multiple choice reading comprehension 
tests . 



CONCLUSION 

Remedial college students trained in discerning question 
answer relations were more successful on tests of reading 
comprehension as measured by multiple choice questions than 
students trained in either summarizing or item writing. 
Developing a sensitivity among students to information sources 
for answering comprehension questions may prove to be effective 
instruction in our preparatory course for the exit exam. 



15 



Brozo, Stahl, Gordon, 14 

REFERENCES 

Anderson, R.C., & Biddle, W.B. (1975) On asking people questions 

about what they are reading. In G.H. Bower (Ed.), 

Psychology of learning and motivation (Vol. 9). 

New York: Academic Press. 
Anderson, T.H., & Armbruster, B.B. (1980). Studying (Tech. Rep. 

No. 155). Urbana: University of Illinois, Center for the 

Study of Reading. 
Andre, M. , & Anderson, T.H. (1978-1979). The development and 

evaluation of a self-questioning study technique. 

Reading Research Quarterly , 14 , 605-623. 
Andre, T. (1979). Does answering higher-level questions while 

reading facilitate productive learning? 

Review of Educational Research , 49 , 280-318. 
Armbruster, B.B., & Brown, A.L. (1984). Learning from reading: 

The role of metacognition. In R. Anderson, J. Osborn & R. 

Tierney (Eds.), Learning to read in American Schools : 

Basal readers and content texts (pp. 273-282). Hillsdale, 

New Jersey: Lawrence Erlbaum Associates. 
BASIC WORD VOCABULARY TEST (1984). Providence, RI: Jamestown 

Publishers . 

Bean, T.W., Singer, H., Sorter, J., & Fr&zee, C. (1983). 
Acquisition of summarization rules as a basis for question 
generation in learning from expository text at the high 
school level. In J. Niles & L. Harris (Eds.), Searches 
for meaning in reading/language processing and instruction — 

I 

! 



16 



Brozo, Stahl , Gordon, 15 
Thirty second yearbook of The National Reading Conference . 
Rochester, N.Y.: The National Reading Conference. 
Brown, A. (1981). Metactognition: The development of selective 
attention strategies for learning from texts. In M. 
Kamil(Ed.), Directions in reading; Research and instruction 
— Thirtieth yearbook of The National Reading Conference , 
(pp. 21-43). Washington, D.C.: The National Reading 
Conference. 

Brown, A.L., Campione, J.C., & Day, J.D. (1981). Learning to 
learn: On training students to learn from text. 
Educational Researcher , 10 , 14-21. 

Brown, A.L., & Day, J.D. (1983). Macrorules for summarizing 
texts: The development of expertise. 
Journal of Verbal Learning and Verbal Behavior , 22 , 1-14. 

Brown, A.L., Day, J.D., & Jones, R.S. (1983). The development of 
plans for summarizing texts. Child Development , 54, 968-976. 

Brozo, W.G. ( 1982). A study of the effect of pre- 

questions on reading comprehension and anxiety for first 
year nursing students . Unpub lished doctoral 
dissertation , University of South Carolina, Columbia, 
South Carolina. 

Cohen, R. (1983). Self-generated questions as an aid to reading 

comprehension. Reading Teacher , 36, 770-775. 
Day, J.D. (1980). Training summarization skills: A comparison 

of teaching methods . Unpublished doctoral 

dissertation University of Illinois. 



Brozo, Stahl, Gordon, 16 
Duell, O.K. (1978). Overt and qovert use of objectives of 
different cognitive levels. Contemporary Journal of 
Educational Psychology , 3_' 239-245. 

Flavell, J.H., & Wellman, H.M. (1977). Metamemory. In R.V. Kail, 

i 

Jr. & w. Hagen (Eds.), Perspectives on the development 

of memory and cognition . Hillsdale, New Jersey: Lawrence 

Erlbaum Associates. 
Gordon, B. (1982). Teach them to read the questions. Journal 

of Reading , 26, 126-131. 
Johnston, P.H. (1981). Prior knowledge and reading 

comprehension test bias . Unpublished doctoral dissertation, 

University of Illinois. 
King, J.R., Biggs, S., & Lipsky, S. (in press). Students' self- 
questioning and summarizing as reading stud' strategies. 

Journal of Reading Behavior . 
Kintsch, W., & van Dijk, T. (1978). Toward a model of text 

comprehension and production. Psychological Review , 85 , 

363-394. 

Pearson, P.D., & Johnson, D.D. (1978). Teaching reading 

comprehension . New York: Holt, Rinehart and Winston. 
Raphael, T.E. (1984). Teaching learners about sources of 

information for answering comprehension questions. Journal 

of Reading , 28, 303-311. 
Rothkopf, E.Z. (1965). Some theoretical and experimental 

approaches to problems in written instruction. In L. 

Krumboltz (Ed.), Learning and the educational process 



18 



Brozo, Stahl, Gordon, 17 
(pp. 193-221). Chicago: Rand-McNally . 
Singer, H., & Donlan, D. (1982). Active comprehension; Problem- 
solving schema with question generation for comprhension of 
complex short stories. Reading Research Quarterly , 17, 166- 
186. 

Stahl, N.A., Brozo, W.G., & Gordon, B. (1983). Metacognitive 
training for improving reading test performance of college 
remedial readers. Unpublished manuscript, Georgia State 
University . 

Trabasso, T. (1981). On the making of inferences during reading 

and their assessment. In J. Guthrie (Ed.), Comprehension 

and teaching: Research reviews (pp. 56-76). 

Newark, Delaware: International Reading Association. 
Tulving, E., & Thomson, D . M .. (1973). Encoding specificity and 

retrieval processes in episodic memory. Psychological 

Review , 80, 352-373. 
van Dijk, T., & Kintsch, W. (1978). Cognitive psychology and 

discovery, in W.U. Dressier (Ed.), Trends in text 

1 inguistics . New York: DeGruyter. 
Winograd, P.N. (1984). Strategic difficulties in summarizing 

tex\_s. Reading Research Quarterly , 19, 404-425. 
Wong, B.Y., & Jones, W. (1982). Increasing metacomprehension in 

learning disabled and normally achieving students through 

self -questioning training. Learning Disability Quarterly , 

5, 228-240. 



19 



TABLE 1 

Results for Experimental Groups 
on the Practice Form* 
Group n Mean Standard deviation 

QAR 17 36.88 7.66 

IW 16 31.69 6.81 

SW 16 29.94 6.54 

*Max. raw score=58; cut-off =37 
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TABLE 2 

Results for Experimental Groups 
on the Regent's Reading Test* 
Group n Mean standard deviation 

QAR 17 61.24 4.76 

IW 16 ' 61.19 3.85 

SW 16 58.13 4.02 

*Max. scale score=99; cut-off =61 
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